Field of the Invention
The present invention relates to a method and apparatus for acquiring a three-dimensional position information of an observed object.
Description of the Related Art
A microscope is an apparatus used to observe an object. The optical system of an microscope typically includes, an objective lens, an aperture stop, an tube lens, and an image pickup element arranged in order from the object side. Moreover, the optical axis of the objective lens and the optical axis of the tube lens are aligned, and the center of the aperture of the aperture stop is also aligned with the optical axis.
FIGS. 1A, 1B, and 1C are diagrams showing states of light beams in a typical optical system as described above. FIG. 1A is a diagram showing a state of converging light beams, and FIGS. 1B and 1C are diagrams showing states of a point spread function formed on the image pickup element.
When light from an object point on the optical axis is focused by the optical system, the light is focused at a position P2 as indicated by broken lines in FIG. 1A. Theoretically or geometrical optically, the image formed at the position P2 is a point. However, the image is not a point actually. As illustrated by solid lines in FIG. 1A, the image has spread to some extent due to diffraction. Here, the image which has such spread will be referred to as a point spread function.
When a displacement between the position of an observed object and the in-focus position of the optical system is generated, spread in the point spread function changes. However, if an amount of displacement of the observed object from the in-focus position of the optical system corresponds to an amount which is smaller than the image side depth of focus of the optical system, the spread in the point spread function does not change largely. The amount of displacement of the observed object from the in-focus position of the optical system will be referred to as an object side displacement amount. The depth of focus on the image side will be simply referred to as a depth of focus.
In FIG. 1A, a region in which the spread of the point spread function does not change largely is represented as the hatched area. In FIG. 1B, the spread of the point spread function is represented by the hatched area. Moreover, a predetermined pixel region is defines as a 4×4 pixel area.
When the position of the observed object and the in-focus position of the optical system coincide with each other, the light from the observed object is focused at the position P2. On the other hand, when the position of the observed object is displaced from the in-focus position of the optical system, the light from the observed object is focused at a position displaced from the position P2 toward a position P1 or a position P3. In the case where the object side displacement amount corresponds to an amount which is smaller than the depth of focus, the image of the observed object is formed between the position P1 and the position P3.
In the case where the object side displacement amount corresponds to an amount which is smaller than the depth of focus, the spread of the point spread function is kept within the predetermined pixel area at each of positions P1, P2, and P3 as shown in FIG. 1B. More specifically, the circle representing the spread of the point spread function is inscribed in the predetermined pixel area.
All of positions P1, P2, and P3 are positions which are located within the depth of focus. Therefore, in the case where the object side displacement amount corresponds to an amount which is smaller than the depth of focus, a change in the object side displacement amount does not appear as a change in the brightness of the point spread function or a change in the brightness of the image of the observed object.
In cases where there is a projection or a recess on the surface of the observed object, the height of the projection or the depth of the recess corresponds to the object side displacement amount. Therefore, in cases where the height of the projection or the depth of the recess on the surface of the observed object corresponds to an amount which is smaller than the depth of focus, a change of the projection or the recess on the surface of the observed object does not appear as a change in the brightness of the point spread function or a change in the brightness of the image of the observed object.
Moreover, at all of positions P1, P2, and P3, the point spread function is located at the same position on the image pickup element. Therefore, in the case where the object side displacement amount corresponds to an amount which is smaller than the depth of focus, a change in the object side displacement amount does not appear as a change in the position of the point spread function or a change in the position of the image of the observed object.
As described above, the object side displacement amount corresponds to the height of a projection or the depth of a recess. In cases where the height of the projection or the depth of the recess corresponds to an amount which is smaller than the depth of focus, a change of the projection or the recess on the surface of the observed object does not appear as a change in the position of the point spread function or a change in the position of the image of the observed object. Therefore, in cases where the height of the projection or the depth of the recess on the surface of the observed object corresponds to an amount which is smaller than the depth of focus, the height of the projection or the depth of the recess cannot be detected by an ordinary optical system.
A defocus amount is defined as the length d of the hatched region in FIG. 1A. The depth of focus, which is determined by the pixel size, is represented by the area in which the circle representing the spread of the point spread function does not exceed the predetermined pixel area (4×4 pixel area). As shown in FIG. 1A, the radius φ of the point spread function is expressed by the following equation (1):φ=(d/2)×NA′  (1),where
NA′ is the numerical aperture of the imaging optical system on the image side.
The resolution of the image pickup element can be expressed by the size of the pixels. The inverse of the resolution of the image pickup element is equal to the size of the two pixels. When the size of one pixel is dpix, the size of two pixels is 2dpix. Since it is possible to replace φ to the size of two pixels, the defocus amount can be expressed by the following equation (2):d=4 dpix/NA′  (2).
Moreover, the resolution of the imaging optical system is expressed by the following equation (3):δOBS=λ/(2×NA′)   (3),where
λ is the wavelength, and
δOBS is the resolution of the imaging optical system.
In the case where the resolution of the imaging optical system and the resolution of the image pickup element are equal to each other, in other words, in the case where the cut-off frequency of the imaging optical system and the Nyquist frequency of the image pickup element are equal to each other, since it is possible to replace δOBS to the size of two pixels, the resolution of the imaging optical system can be expressed by the following equation (3′):2dpix=λ/(2×NA′)   (3′).
From equations (2) and (3′), the defocus amount can be expressed by the following equation (4):d=λ/NA′2   (4).
Since the depth of focus of the imaging optical system is represented by λ/NA′2, the depth of focus determined by the pixel size is equal to the depth of focus of the imaging optical system. Therefore, in the case where the resolution of the imaging optical system and the resolution of the image pickup element are equal to each other, the depth of focus of the observation system can be represented by the depth of focus of the imaging optical system or the depth of focus determined by the pixel size.
In the case where the resolution of the imaging optical system is lower than the resolution of the image pickup element, the point spread function is sampled by an area larger than the predetermined pixel area. In this case, the depth of focus of the observation system can be represented by the depth of focus of the imaging optical system, that is, λ/NA′2.
In the case where the imaging optical system is an optical system of a microscope, the magnification of the imaging optical system changes with replacement of the objective lens. Since the numerical aperture NA′ also changes with the change in the magnification, the resolution of the imaging optical system also changes. A decrease in the magnification of the imaging optical system with replacement of the objective lens may make the numerical aperture NA′ larger than that in the state in which the magnification is high, in some cases. When the numerical aperture NA′ becomes larger, the resolution of the imaging optical system becomes higher than the resolution of the image pickup element.
Here, we will discuss a case where the resolution of the imaging optical system is higher than the resolution of the image pickup element and a case where the resolution of the imaging optical system and the resolution of the image pickup element are equal to each other in comparison. Firstly, comparison of the depth of focus of the imaging optical system will be discussed.
The numerical aperture NA′ is larger in the case where the resolution of the imaging optical system is higher than the resolution of the image pickup element than in the case where the resolution of the imaging optical system is equal to the resolution of the image pickup element. Therefore, the depth of focus of the imaging optical system is smaller in the case where the resolution of the imaging optical system is higher than the resolution of the image pickup element than in the case where the resolution of the imaging optical system is equal to the resolution of the image pickup element.
Next, comparison of the defocus amount, which determines the depth of focus, will be discussed. As will be seen by comparison of FIG. 1B and 1C, the spread of the point spread function is smaller in the case where the resolution of the imaging optical system is higher than the resolution of the image pickup element than in the case where the resolution of the imaging optical system is equal to the resolution of the image pickup element. As described above, the depth of focus determined by the pixel size is the range in which the circle representing the spread of the point spread function does not exceed the predetermined pixel area. Therefore, the depth of focus determined by the pixel size is larger in the case where the resolution of the imaging optical system is higher than the resolution of the image pickup element than in the case where the resolution of the imaging optical system is equal to the resolution of the image pickup element.
From the above follows that in the case where the resolution of the imaging optical system is higher than the resolution of the image pickup element, the depth of focus determined by the pixel size is larger than the depth of focus of the imaging optical system. Therefore, in the case where the resolution of the imaging optical system is higher than the resolution of the image pickup element, the depth of focus of the observation system is represented by the depth of focus determined by the pixel size rather than the depth of focus of the imaging optical system.
The depth of focus of the observation system is larger in the case where the resolution of the imaging optical system is higher than the resolution of the image pickup element than in the case where the resolution of the imaging optical system is equal to the resolution of the image pickup element. Therefore, there is a possibility that the object side displacement amount that can be detected in the case where the resolution of the imaging optical system is equal to the resolution of the image pickup element cannot be detected in the case where the resolution of the imaging optical system is higher than the resolution of the image pickup element.
There are apparatuses that acquire various information of an observed object, which include apparatuses that utilize the entire light from an observed object and apparatuses that utilize a part of the light from an observed object. An apparatus that utilizes a part of the light from an observed object is disclosed in Japanese Patent Application Laid-Open No. 2013-235110.
The apparatus disclosed in Japanese Patent Application Laid-Open No. 2013-235110 is an auto-focus apparatus having an optical system that is different from the above-described typical optical system. In this apparatus, a plurality of apertures are arranged at a position substantially conjugate with the position of the exit pupil of the objective lens system. The apertures are arranged at positions away from the optical axis of the objective lens system. This apparatus can calculate the amount of displacement from the in-focus position. Therefore, it is possible to obtain height information of an observed object by using this apparatus.